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[57] ABSTRACT 

Static fade in a digital video image is detected. A difference,, fit \ 
in luminance intensity is determined for correspondingly 
positioned pixels in successive images. A varianc e in the p*sK*fA*£t 
difference is determined, and then normalized accSfSing to ^ 
a mean squared value of the difference to provide a fade 
index J„. The fade index is compared to a fade threshold, 
e.g., 0.9, to determine if a fade is indicated. Optionally, a 
check may be made to verify that a static fade is indicated 
by comparing the mean squared value of the difference to a 
motion threshold. The technique is suitable for both frame 
mode and field mode video sequences. When the digital 
images include field mode video, where each image has first 
and second fields, a fade index is determined separately for 
each field, and an overall fade index is obtained by averag- 
ing the fade indexes from each field. A scheme for detecting 
static fade on a macroblock by macroblock basis is also 
presented. 

26 Claims, 5 Drawing Sheets 
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FADE DETECTOR FOR DIGITAL VIDEO 
BACKGROUND OF THE INVENTION 

The present invention relates to a method and apparatus 
for detecting static fade in a digital video image such as a 
video frame. A method and apparatus for detecting static 
fade for individual macroblocks in a video image is also 
presented. In particular, the invention is suitable for detect- 
ing temporal fluctuations of ambient light intensity in a 
source video image. With reliable detection of a fade, coding 
efficiency can be improved. 

Recently, digital video transmission formats have become 
increasingly popular for providing television and other 
audio, video, and/or data services to consumers' homes. 
Such programming can be transmitted to a consumer's home 
via a direct broadcast satellite link, cable television network, 
telephone network, or other broadcast means. In addition to 
providing new and expanded program offerings, digital 
video can provide a higher image and audio quality than 
traditional analog television broadcasts. 

In order to transmit digital video signals within available 
bandwidths, data compression is necessary. In particular, 
spatial compression techniques take advantage of correla- 
tions between neighboring pixels or blocks of pixels in a 
frame, while temporal techniques take advantage of corre- 
lations between pixels or blocks of pixels in successive 
video frames. Moreover, motion compensation techniques 
can provide even greater temporal compression. 

However, to provide optimal compression of a sequence 
of video frames, it is desirable to have the capability to 
detect fade in or fade out in the image sequence. Fade in 
refers to a gradual increase in light intensity over two or 
more frames or other images, while fade out refers to a 
gradual decrease in light intensity. Fades may be used by 
cinematographcrs and other video editors to provide a 
dramatic effect or to emphasize changes in a story line, for 
example. 

Current motion estimation hardware in digital video 
encoders such as those conforming to the MPEG-2 standard 
utilizes block matching algorithms with the objective of 
minimizing the sum of absolute error in the prediction. 
However, this approach fails to cope with temporal variation 
of light intensity such as fade in and fade out in the source 
video. Details of the Motion Picture Expert's Group 
(MPEG) standard can be found in document ISO/IEC JTC1/ 
SC29/WG11N0702, entitled "Information Technology- 
Generic Coding of Moving Pictures and Associated Audio, 
Recommendation H.262," Mar. 25, 1994. 

It is desirable to detect a static fade, that is, a fade in or 
fade out effect on a still scene or on a scene that contains a 
small amount of motion, so that the video can be more 
efficiently coded. In particular, when a static fade is detected, 
the motion vectors can be forced to be zero, so less data is 
encoded and transmitted. 

Various methods have been used to detect fade. For 
example, the sum of luminance pixel values may be com- 
puted for each frame. A scene is then classified as a fade if 
frame-to-frame changes in the sum exceeds a preset thresh- 
old. In another approach, the sum of luminance pixel dif- 
ferences between a current frame and the previous frame is 
computed. A scene is then classified as a fade if this sum 
exceeds a preset threshold. These methods are mathemati- 
cally equivalent, although the implementation is different, 
and generally require the same amount of hardware. 
However, these approaches are generally not reliable for 
complicated scenes that contain a large amount of motion. 
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In another approach, a histogram of the pixel differences 
between the current frame and the previous frame is calcu- 
lated. Fade is then detected by inspecting the histogram. This 
approach is more reliable, but the hardware required to 
5 implement a histogram calculator is relatively complex and 
expensive. 

Accordingly, it would be desirable to have a method and 
apparatus for detecting fade in a video image sequence that 
is relatively simple to implement with low cost hardware, 
10 and requires only minor modifications to existing telecine or 
television detection hardware. The scheme should avoid 
false fade indications by distinguishing non-fade scenes 
such as stills and pictures that contain slow moving objects. 

The scheme should further be compatible with existing 
15 digital video communication systems, including MPEG-1, 
MPEG-2, MPEG-4, ISO/IEC H.261 (videoconferencing), 
and ISO/IEC H.263. Moreover, the system should also be 
compatible with various color television broadcast standards 
such as the National Television Standards Committee 
20 (NTSC) standard, used in North America, and the Phase 
Alternating Line (PAL) standard, used in Europe, and should 
further be compatible with both frame and field mode video. 

It would further be desirable to have a scheme for 
25 detecting static fade on sub-regions of a video image, such 
as on a macroblock by macroblock basis in a video frame, 
and setting the corresponding motion vectors to zero. 

The present invention provides a system having the above 
and other advantages. 

30 

SUMMARY OF THE INVENTION 

In accordance with the present invention, a method and 
apparatus are presented for detecting static fade in a digital 
video image. 

35 A method for detecting a fade in a sequence of digital 
video images includes the step of determining a difference d„ 
in pixel luminance intensity between first and second 
images. The difference in luminance intensity is determined 
for pixels which are positioned in corresponding (x.y) loca- 
tions in the first and second images. A variance V n in the 
difference is determined, and then normalized according to 
a mean squared value E„/N of the difference to provide a 
fade index J„. The fade index is compared to a fade threshold 
to determine if a fade is indicated. For example, if J_<0.9, a 

45 

fade is indicated. 

Optionally, a check may be made to verify that a static 
fade is indicated. This is accomplished by comparing the 
mean squared value of the difference to a motion threshold. 

5 q For example, testing has shown that the motion threshold 
may be 256 times the number of pixels in a field or frame, 
assuming pixel values range from 0 to 255. If the mean 
squared value of the difference is below the motion 
threshold, a static fade is indicated. Thus, the final decision 

55 of whether a static fade is indicated may depend on both the 
fade threshold and the motion threshold. 

The technique is suitable for both frame mode (e.g., 
non-interlaced, or "progressive") and field mode (interlaced) 
video sequences. When the digital images include field 

go mode video, where each image has first and second fields, 
the difference in luminance intensity is determined sepa- 
rately for each field of the first and second images. For 
example, d nl may represent the first field differences, while 
d„2 represents the second field differences. 

65 The variance in the differences is also determined sepa- 
rately for each field, and the variances are normalized 
according to a mean squared value of the differences to 
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provide a fade index for each field. For example, J ttl may easily implemented with only one extra accumulator, 

represent the first field fade index, while J„ 2 represents the Moreover, the effectiveness of the fade detection algorithm 

second field fade index. An overall averaged fade index is has been verified by simulations on a number of standard test 

then determined for each of the images over the fields, e.g., sequences. 

J a «=( J ni +J rt2) /2- and tne averaged fade index is compared 5 . , . 

to me fade^reshold to determine if a fade is indicated. . A video segment of static fade out can be modeled as a 

. , , still image which is modulated by a sequence of gam factors 

A corresponding apparatus is also presented. { a* a*,...} such that, for field mode (e.g., interlaced) 

A method is also presented for detectmg fade for sub- video> ±e Iuminailce value ^ frame coordinate (x,y) of the 

regions such as macroblocks in a digital video image. An n|h . ( field fc 

overall frame to frame residue is determined between a 

current frame macroblock and a correspondingly situated 

previous frame macroblock. This residue is used to deter- 
mine an overall frame to frame variance of the current P&y><iJ'n-&y) 

macroblock. An overall motion compensation residue 

between the current macroblock and a best match macrob- 15 where P„_ 2 (x,y) denotes the luminance values of the (n-2)th 
lock in the previous frame is also determined, and used to input field, and O^a^ 1. For fade out, a decreases from one 
provide an overall motion compensation variance V mc of the to zero with time or frame number. For fade in, a increases 
current macroblock. Astatic fade is indicated if V me >V jr from zero to one with time or frame number. Similarly, for 
For field mode video, an intermediate frame to frame 2Q frame mode (e.g., progressive) video, the luminance value at 
residue and intermediate frame to frame variance are deter- frame coordinate (x,y) of the nth input frame is 
mined separately for the top and bottom fields of the current 
macroblock using the corresponding fields of the corre- 
spondingly situated macroblock of the previous frame. An PJ.x,y)-aJ'^ l {x,y). 
intermediate motion compensation residue and intermediate 25 

motion compensation variance are also determined sepa- Io me casc Q f f at j c m> ^th field mode, P^x.yMl/a^ 

rately for the top and bottom fields of the current macroblock p ^ Xp y). for example, assume P 20 (x,y) is the final steady 

using the corresponding fields of the best match macroblock ^ m a fadc ^ y^n, an example prior pixel value is 

of the previous frame. Finally, the intermediate frame to ( y)=(l/a 16 ) (l/a ie ) (l/a^P^x.y). With frame mode, 

frame variance is averaged over the top and bottom fields to 3Q p £ vWlAx \P 

obtain the overall frame to frame variance, and the inter- ^^Wl 1 /"^ 

mediate motion compensation variance is averaged over the Ideally, static fade can be detected by performing a 

top and bottom fields to obtain the overall motion compen- statistical analysis of the ratio r H (x,y)-P„(x,y)/P^ 2 f> yr) for 

sation variance everv ^P 111 P ixeL Id P 3 ^ 01113 ^ ^ 1D P ut satisfies the 

. t , , fade model described above, the ratio r_(x,y) will be a 

A corresponding apparatus is also presented. 35 constam which fc equal to ^ for every pixel m me nth input 

BRIEF DESCRIPTION OF THE DRAWINGS field. For frame mode video, r„(x,y) will ideally be equal to 

a > _ 1 . However, there arc drawbacks to this approach. First, 

FIG. 1 is a block diagram of a fade detector for a video mc computation of the ratio of the pixels in real time 

frame in accordance with the present invention. requires significant hardware complexity. Secondly, since a 

FIG. 2 is a graph showing the fade index for the "Flower- 40 function of the form 1/x is sensitive to changes in x for small 

fade" video test sequence in accordance with the present x , a statistical analysis of the ratio r„(x,y) is sensitive to 

invention. source noise, truncation error, and deviations from the 

FIG. 3 is a graph showing the fade index for the "Coo- idealized static fade model, 

fetti" video test sequence in accordance with the present ^ A simpler and more robust scheme to detect static fade is 

invention. therefore needed. A fade detection technique in accordance 

FIG. 4 is a graph showing the fade index for the "Flower with the present invention detects fade by performing a 

garden" video test sequence in accordance with the present statistical analysis of the change in luminance intensities 

invention. with respect to the previous frame or field. For field mode 

FIG. 5 is a block diagram of a fade detector for an 50 video, the change in luminance intensity for pixels in the nth 

individual macroblock of a video image in accordance with odd or even field is given by 
the present invention. 

DETAILED DESCRIPTION OF THE d ^yp^yp^y). 

INVENTION ss 

A method and apparatus are presented for detecting static For frame mode video, the change in luminance intensity for 

fade in a digital video image. A simple and low cost pixels in the nth frame is given by 
technique detects fade, or, in general, global changes in light 
intensity of source video. 

Fade is detected by inspecting the degree of uniformity of 60 dj^yyp^yyp^^y). 
the difference image between current and previous input 

frames. The degree of uniformity is measured by the It is assumed that, during a fade, the luminance differ- 

variance, normalized by the mean squared value of the ences cl^y) are approximately uniform over the field of 

differences of the luminance pixel values over the picture. input video so that a variance (V„) of d„(x,y) oyer the field 

The fade detector shares the mean squared error calculator 65 is small. Based on this assumption, static fade in the video 

that is being used by many existing telecine detectors and could be detected by comparing the variance of the lumi- 

secne change detectors. Therefore, the fadc detector can be nance differences with a preset threshold. However, it was 
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found that the variance is also small in some non-fade scenes 
such as stills and pictures that contain slow moving objects. 

Therefore, to avoid falsely classifying these scenes as 
fades, the variance of the luminance difference is normalized 
by the mean squared value of the luminance difference to 
generate a fade index J„ for the field or frame of input video, 
namely 


(EJN) 


where 


D m = £ 4,0c, y), 


and N is the number of pixels in the field or frame. For frame 
mode video, (x,y)e Frame for E„ and D„. Accordingly, if a 
field or frame has a low fade index value, it is very likely that 
the field or frame belongs to a fade. E„ is the sum of the 
squared luminance differences, and D„ is the sum of the 


divisor function 150. The divisor function 150 provides the 
fade index for the present field J„=V n /(E,/N) to a field delay 
function 170 and a summer and divisor function 160. The 
field delay function 170 is a temporary storage element such 
as a random access memory (RAM) which stores the first 
field fade index while a second field of the image is being 
processed. When the fade index for the second field has been 
output from the divisor function 150 and provided to the 
summer and divisor function 160, the summer and divisor 
10 function 160 takes the average of the fade indexes for the 
first and second fields. For example, let 3 nl and be the 
first and second field fade indexes, respectively. Then an 
average or overall fade index is hve'iKi^K^- 
After the fade index is averaged over both fields of the 
is frame, it is compared to an empirical threshold at a detector 
(e.g., comparator) 165. A fade is indicated if the average is 
below the threshold. As discussed below and in connection 
with FIGS. 2-4, a threshold value of 0.9 can correctly 
identify most fade sequences. 
20 When the motion detector 180 and AND gate 175 are 
provided in an optional embodiment, a signal is provided 
from the detector 165 to the AND gate 175, and a corre- 
sponding static fade flag is set. That is, if J„ is below the fade 
threshold and E„/N is below the motion threshold, a fade is 


luminance differences. _ _ 

For a sequence that satisfies the above ideal static fade 25 .T^^pE^e" t&^£ J£T* toto is 


model, it can be easily shown that 


indicated. The fade flag is used to set the motion vectors to 


j„ = 


fi 2 + V n 


While the fade detector of FIG. 1 has been discussed in 
connection with a 2:1 interlaced video image, it will be 
30 understood that it may be adapted for other interlaced and 
pulldown video formats. Additionally, frame mode video 
may be processed as shown in FIG. 1, with the following 
changes. First, the accumulators 125 and 130 accumulate 
values over one frame instead of one field. Second, the delay 
170 and average 160 blocks are not needed. Third, the value 
N in the divisor function 140 is taken over the entire frame 
instead of field. 

Additionally, the scheme may be adapted for use with 
portions of a frame, such as a video object plane (VOP) as 


35 


where fi is the mean and V„ is the variance of the image that 
generates the fade sequence (i.e., the image at the start of the 
fade sequence). For most fade images, the magnitude of /j 2 
is comparable to that of V„, so J„ is well below 1. On the 
other hand, for a typical motion video without fade, the 
luminance differences are equally likely to have positive and 
negative values, so J„ is close to one. This is true since the 
magnitude of (D^/N) 2 is small compared to (E„/N). 

FIG. 1 is a block diagram of a fade detector for a video 
frame in accordance with the present invention. The fade 40 defined in the MPEG-4 standard, 
detector may be used as part of a digital video encoder. Input Moreover, it will be appreciated that various other hard- 
video is provided to a terminal 106 and a one frame delay ware and/or firmware implementations are possible. For 
function 110. A difference between the delayed frame and example, a parallel processing configuration may be used to 
the input video is taken at a subtractor 115 and provided to process the first and second fields of an interlaced video 
a square function 120. The square of the difference is 45 image at the same time. 

provided to an accumulator 125 which accumulates the The fade detection technique of the present invention has 
squared difference values over one field to provide E„, the been tested on sequences with varying degrees of motion, 
sum of squared differences. and has been shown to correctly identify the fading scenes. 

The output of the subtractor 115 is also provided to an However, forcing zero- motion should apply only to static 
accumulator 130, which accumulates the difference values 50 fade, that is, fade on a picture sequence that contains an 
to provide D„, the sum of pixel differences. For every input insignificant amount of motion. To avoid forcing motion 
field, E„ and D„ are read by a host central processing unit vectors to zero for fading scenes with a large amount of 
(CPU) 135. The"host CPU may be implemented in firmware motion, the fade detector of FIG. 1 optionally uses the 
using digital signal processing techniques. In particular, E„ motion detector 180 and AND gate 175. It can be assumed 
is divided by N, the number of pixels in the field, at a divisor 55 that there is an insignificant amount of motion when the 


function 140 to provide the mean square error, E„/N to a 
subtractor 145. Advantageously, hardware for calculating 
the mean square error is already used in many existing 
telecine detectors and scene change detectors, so the accu- 
mulator 130 may be the only extra hardware required. E„/N 60 
is also optionally provided to a motion detector (e.g., 
comparator) 180 for comparison with a motion threshold, as 
discussed below. A corresponding signal is then provided to 


mean squared value of the differences between successive 
frames or fields, E„/N, is below a motion threshold. Note 
that the value E,/N is readily available in the fade detector 
for input to the comparator 180. 

Table 1, below, shows the result of fade detection for test 
sequences in which the threshold on the fade index is 0.9, 
and the motion threshold for the average sum of squared 
difference EJN is 256 times the number of pixels N in a 
frame (for frame mode video) or field (for field mode video), 


an AND gale 175. 

D„ is divided by N, and the resulting value, D„/N, is 65 assuming the pixel values range from 0 to 255. This empiri- 

squared at a function 155. At the subtractor 145, the differ- cal value has been shown in experiments to be effective. The 

ence E„/N-(D„/N) 2 is taken to provide the variance V H to a test sequences in Table 1 are described in Test Model Editing 
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Committee, "Test Model 5*\ ISO/IEC JTC1/SC29/WG11 
MPEG 93/457, April 1993. The scheme was also shown to be 
effective with test sequences where realistic levels of noise 
were introduced. 

TABLE 1 


Sequence 

Fade Delected 

Static Fade Detected Comments 

Starsailer- 

frames 0-29 

frames 0-29 

fade 



Flower-fade 

frames 0-29 

frames 0-29 

Confetti 

frame 111-119 

frames 118-119 The brightness 



of frames 118- 



119 is close 



to black. 

Flower garden 

none 

none 

Bus 

none 

none 

Mobile- 

none 

none 

calendar 



Cheer-leader 

□one 

none 

NHK-flower 

none 

none 


The NHK-flower sequence contains slowly moving 
objects. For this sequence, both the variance and the mean 
squared values of the luminance differences are small. 
However, the fade indices are very close to one and the 
sequences arc correctly classified. 

A statistical analysis of the luminance pixel differences 
for a number of test sequences was performed The values of 
the fade index J„ for field mode video averaged over both 
fields for each input frame are plotted in FIGS. 2-4. As seen 


selected as the macroblock in a defined search error which 
results in the smallest error between the current macroblock 
and the previous frame macroblock, e.g., according to the 
sum of the absolute differences or other criteria used in 
5 known video coding standards. 

Next, calculate the variance V # of a frame to frame 
difference, that is, a motion compensated residual with the 
motion vector set to zero for frame motion. This residual is 
the difference in pixel luminance values between a current 
10 frame macroblock and the macroblock in the previous frame 
which is positioned in a corresponding location in the frame. 

For field mode video, the variances of top and bottom 
fields are calculated separately and then averaged to gener- 
ate the variance of the macroblock. For example, a variance 
15 V^..^ is determined for the residual between the top field 
of the current frame macroblock and the top field of the best 
match macroblock in the previous frame. Likewise, a vari- 
ance V 'wbonem is determined for the residual between the 
bottom field of the current frame macroblock and the bottom 
20 field of the best match macroblock in the previous frame. 
Then, the variance of the macroblock is equal to 


(y^c-^mc-boaom)^- V mc-co P and V^***, may be con- 
sidered to be intermediate motion compensation variances, 
while V mc is an overall motion compensation variance. 
25 Similarly, the variance Vy for field mode video can be 
determined as where is the frame 

to frame variance for the top field and V-^,,^ is the frame 
to frame variance for the bottom field. v^, op and V^..^^ 
may be considered to be intermediate frame to frame 
from the measured data, a threshold value of 0.9 can 30 variances, while V^- is an overall frame to frame variance, 
correctly classify most frames. In accordance with the present inveotion, a static fade is 

FIG. 2 is a graph showing the fade index for the "Flower- indicated for each macroblock in the current frame when 
fade" video test sequence in accordance with the present V^V^c- Tne ^heme nas been shown to be effective with 
invention. Frame number is shown on an axis 200, while the testing of various video sequences. Alternatively, a bias or 
value of the fade index is shown on an axis 210. The fade 35 offset may be introduced such that a static fade is indicate 
index J„ is approximately 0.2, which is well below the when V^cV^+ki, or when V^^V^+l^, or the like, 
threshold of 0.9. Accordingly, a fade is indicated for the where k t and k 2 are constants. When a static fade is 
frames shown. indicated for a macroblock, the motion vector for the mac- 

F1G. 3 is a graph showing the fade index for the "Con- roblock is set to zero. The scheme may be repeated for each 
fetti" video test sequence in accordance with the present 40 macroblock in the video image of the current frame until 
invention. Frame number is shown on an axis 300, while the each macroblock has been processed. For macroblocks 
value of the fade index is shown on an axis 310. The Confetti where a static fade is not indicated, the non-zero motion 
sequence has a number of randomly flying objects in a bright vector will be transmitted. Coding efficiency is improved by 
background. It fades into black in the last 10 frames, setting the motion vector to zero for at least some of the 
beginning at approximately frame 111. The fade out is 45 macroblocks in a video frame. 


clearly identified in the plot where J„ falls below 0.9. 

FIG. 4 is a graph showing the fade index for the "Flower 
garden" video test sequence in accordance with the present 
invention. Frame number is shown on an axis 400, while the 
value of the fade index is shown on an axis 410. The 50 
Flower-garden is a typical bright flowery video sequence 
with no fade indicated. Accordingly, note that the values of 
the fade index J„ is close to one throughout the sequence, 
well above the fade threshold. 


While the above scheme has been discussed in terms of 
macroblocks such as 16x16 luminance macroblocks, it will 
be appreciated that the scheme may be adapted for use with 
other sized sub-regions of a video image. 

The scheme shown in FIG. 5 is suitable for implementing 
the macroblock by macroblock processing scheme of the 
present invention. A motion compensator 505 receives 
motion vectors (MVs) from a motion estimator which indi- 
cate the best match macroblocks in a reference frame (e.g., 


FIG. 5 is a block diagram of a fade detector for an 55 the previous frame). A field/frame decision is provided to 


individual macroblock of a video image in accordance with 
the present invention. In this alternative embodiment of the 
present invention, coding efficiency can be improved in fade 
scenes by selectively setting motion vectors to zero for 
different sub-regions of a video frame, such as on a mac- 
roblock by macroblock basis. 

First, for each macroblock, calculate a variance of the 
motion compensated residual with motion vectors generated 
by conventional motion estimation. This residual is the 


indicate whether the video is field or frame mode. The 
motion compensator 505 also receives a data signal com- 
prising the reference frame pixel data, and provides a signal 
comprising the best match macroblock to a subtractor 520. 
60 The subtractor 520 also receives a signal comprising the 
current macroblock data from a current frame memory 525 
and calculates a corresponding residue separately for the top 
and bottom fields for field mode video. For frame mode 
video, only one residue is calculated by the subtractor 520. 


difference in pixel luminance values between a current 65 The top field residue is provided to a top field variance 
frame macroblock and the best match macroblock in the calculation function 535, which calculates the variance 
previous frame. The best match macroblock is typically V mc . (cy o(E„/N)-(D„/N) 2 . Similarly, the bottom field residue 
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is provided to a bottom field variance calculation function 
540, which calculates the variance .V^^^«(E„/N)-( I V 
N) 2 . An average of V^.,^ and V^*^ (i.e., V m J is 
determined at an averaging function 555 and provided to a 
decision function (e.g., comparator) 560. 

A motion compensator 515 is similar to the motion 
compensator 505 but uses a motion vector which is fixed to 
zero to provide a signal comprising a non-motion compen- 
sated macroblock from the reference frame to the subtracter 
530. The subtractor 530 also receives a signal comprising 
pixel data from the current macroblock from the current 
frame memory 525, and outputs a residue which is indicative 
of the frame to frame difference between the current mac- 
roblock and the correspondingly positioned reference mac- 
roblock. The residue is calculated separately for the top and 
bottom fields for field mode video. 

The top field frame to frame residue is then provided to a 
top field variance calculation function 545 which corre- 
sponds to the top field variance calculation function 535, and 
to a bottom field variance calculation function 550 which 
corresponds to the bottom field variance calculation function 
540. The top field variance calculation function 545 calcu- 


20 


calculation function 550 calculates the variance V^^^. 
An average of and Vpt^^ (i.e., V ff ) is determined at 
an averaging function 565 and provided to the decision 2 $ 
function 560. 

At the decision function 560, if V me >V« a static fade is 
indicated for the current macroblock, and the motion vector 
for that macroblock may be set to zero. The decision 
function may implement a modified decision criteria which 3Q 
includes the bias terms discussed above. 

Accordingly, it can be seen that the present invention 
provides a method and apparatus for detecting fade in a 
video sequence. A fade index J„ is calculated and compared 
to a fade threshold, such as 0.9. A fade in or fade out is 35 
indicated if J„ is below the threshold. Optionally, the mean 
square difference, EJN is compared to a motion threshold to 
verify that there is little or no motion, so that a still or static 
fade is detected. The scheme can reliably detect fades while 
providing a low cost implementation which utilizes hard- ^ 
ware already present in most telecine detectors or scene 
change detectors. Various test sequences demonstrated the 
effectiveness of the scheme. 

In an alternative embodiment, a scheme for detecting 
static fade on a macroblock by macroblock basis is pre- 
sented. A frame to frame variance V ff is compared to a 
motion compensated variance for each macroblock to 
determine if a static fade is indicated. 

Although the invention has been described in connection 
with various specific embodiments, those skilled in the art 
will appreciate that numerous adaptations and modifications 
may be made thereto without departing from the spirit and 
scope of the invention as set forth in the claims. 

What is claimed is: 

1. A method for detecting a fade in a sequence of digital J5 
video images, comprising the steps of: 

determining differences in pixel luminance intensity 

between first and second ones of said images; 
determining a variance in said differences; 
normalizing said variance according to a mean squared &o 

value of said differences to provide a fade index; and 
comparing said fade index to a fade threshold, and com- 
paring said mean squared value to a motion threshold, 
to determine if a static fade is indicated; wherein: 
said fade threshold is approximately 0.9; and the static 65 
fade is indicated if said fade index is below said fade 
threshold. 


45 


50 


2. The method of claim 1, wherein said digital images 
comprise field mode video, each image including first and 
second fields, said method comprising the further steps of: 

determining differences in pixel luminance intensity sepa- 
rately for each field of said first and second images; 

detennicing a variance in said differences for each of said 
fields; 

normalizing said variances according to a mean squared 

value of said differences to provide a fade index for 

each of said fields; 
determining an averaged fade index for each of said 

images over said fields; and 
comparing said averaged fade index to a fade threshold to 

determine if a fade is indicated. 

3. The method of claim 1, wherein: 

said differences in pixel luminance intensity are deter- 
mined for pixels which are positioned in corresponding 
locations in said first and second images. 

4. The method of claim 1, wherein: 

said motion threshold corresponds approximately to said 

magnitude of said range; and 
the static fade is indicated when said mean squared value 

is less than said motion threshold. 

5. An apparatus for detecting a fade in a sequence of 
digital video images, comprising: 

means for determining differences in pixel luminance 
intensity between first and second ones of said images; 

means for determining a variance in said differences; 

means for normalizing said variance according to a mean 
squared value of said differences to provide a fade 
index; and 

means for comparing said fade index to a fade threshold, 
and for comparing said mean squared value to a motion 
threshold, to determine if a static fade is indicated; 
wherein: 

said fade threshold is approximately 0.9; and the static 
fade is indicated if said fade index is below said fade 
threshold. 

6. The apparatus of claim 5, wherein said digital images 
comprise field mode video, each image including first and 
second fields, said apparatus further comprising: 

means for determining differences in pixel luminance 

intensity separately for each field of said first and 

second images; 
means for determining a variance in said differences for 

each of said fields; 
means for normalizing said variances according to a mean 

squared value of said differences to provide a fade 

index for each of said fields; 
means for determining an averaged fade index for each of 

said images over said fields; and 
means for comparing said averaged fade index to a fade 

threshold to determine if a fade is indicated. 

7. The apparatus of claim 5, wherein: 

said differences in pixel luminance intensity are deter- 
mined for pixels which are positioned in corresponding 
locations in said first and second images. 

8. The apparatus of claim 5, wherein: 

said motion threshold corresponds approximately to said 

magnitude of said range; and 
the static fade is indicated when said mean squared value 

is less than said motion threshold. 

9. A method for detecting a fade in a current sub-region 
of a current video image in a sequence of digital video 
images, comprising the steps of: 
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determining an overall frame to frame residue between 

said current sub- region and a correspondingly situated 

sub-region of a reference video image; 
determining an overall frame to frame variance of said 

current sub-region according to said overall frame to 

frame residue; 
determining an overall motion compensation residue 

between said current sub-region and a best match 

sub-region of said reference video image; 
determining an overall motion compensation variance of 

said current sub -region according to said overall 

motion compensation residue; and 
determining if a static fade is indicated for said current 

sub-region according to relative magnitudes of said 

overall motion compensation variance and said overall 

frame to frame variance. 

10. The method of claim 9, wherein said digital images 
comprise field mode video, each image including first and 
second fields, said method comprising the further steps of: 

determining a frame to frame residue and intermediate 
frame to frame variance separately for each field of said 
current sub-region using the corresponding fields of 
said correspondingly situated sub-region; 

determining an intermediate motion compensation resi- 
due and intermediate motion compensation variance 
separately for each field of said current sub-region 
using the corresponding fields of said best match sub- 
region; 

averaging the intermediate frame to frame variance over 

each field of said current sub-region to obtain said 

overall frame to frame variance; and 
averaging the intermediate motion compensation variance 

over each field of said current sub-region to obtain said 

overall motion compensation variance. 

11. The method of claim 9, wherein: 

a static fade is indicated for said current sub-region when 
the magnitude of said overall motion compensation 
variance is greater than the magnitude of said overall 
frame to frame variance. 

12. The method of claim 9, wherein: 

a static fade is indicated for said current sub-region when 
the magnitude of said overall motion compensation 
variance is greater than the magnitude of said overall 
frame to frame variance with a bias. 

13. The method of claim 9, wherein: 

said current sub-region comprises a macrobtock. 

14. An apparatus for detecting a fade in a sub-region of a 
current video image in a sequence of digital video images, 
comprising: 

means for determining an overall frame to frame residue 
between said current sub- region and a correspondingly 
situated sub-region of a reference video image; 
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15. The apparatus of claim 14, wherein said digital images 
comprise field mode video, each image including first and 
second fields, further comprising: 

means for determining a frame to frame residue and 
intermediate frame to frame variance separately for 
each field of said current sub-region using the corre- 
sponding fields of said correspondingly situated sub- 
region; 

means for determining an intermediate motion compen- 
sation residue and intermediate motion compensation 
variance separately for each field of said current sub- 
region using the corresponding fields of said best match 
sub-region; 

means for averaging the intermediate frame to frame 
variance over each field of said current sub-region to 
obtain said overall frame to frame variance; and 

means for averaging the intermediate motion compensa- 
tion variance over each field of said current sub-region 
to obtain said overall motion compensation variance. 

16. The apparatus of claim 14, wherein: 

a static fade is indicated for said current sub-region when 
the magnitude of said overall motion compensation 
variance is greater than the magnitude of said overall 
frame to frame variance. 

17. The apparatus of claim 14, wherein: 

a static fade is indicated for said current sub-region when 
the magnitude of said overall motion compensation 
variance is greater than the magnitude of said overall 
frame to frame variance with a bias. 

18. The apparatus of claim 14, wherein: 

said current sub-region comprises a macroblock. 

19. A method for detecting a fade in a sequence of digital 
video images, including first and second images, each image 
including first and second fields, comprising the steps of: 

determining first differences in pixel luminance intensity 
for the first fields of said first and second images; 

determining second differences in pixel luminance inten- 
sity for the second fields of said first and second 
images; 

deteraining a first variance in said first differences, and a 

second variance in said second differences; 
normalizing said first variance according to a mean 

squared value of said first differences to provide a fade 

index for said first fields; 
normalizing said second variance according to a mean 

squared value of said second differences to provide a 

fade index for said second fields; and 
comparing an average of the fade indexes to a fade 

threshold, and comparing an average of the mean 

squared values to a motion threshold, to determine if a 

static fade is indicated. 

20. An apparatus for detecting a fade in a sequence of 
digital video images, including first and second images, each 


means for determining an overall frame to frame variance ss image including first and second fields, comprising: 

- . . . * ■ » » 1 ..11 ^ ^ _ Awl /4ifT A rArt/vir in hiITaI I tin" 


of said current sub-region according to said overall 

frame to frame residue; 
means for detennining an overall motion compensation 

residue between said current sub-region and a best 

match sub-region of said reference video image; 50 
means for detenriining an overall motion compensation 

variance of said current sub-region according to said 

overall motion compensation residue; and 
means for determining if a static fade is indicated for said 

current sub-region according to relative magnitudes of 65 

said overall motion compensation variance and said 

overall frame to frame variance. 


means for determining first differences in pixel luminance 
intensity for the first fields of said first and second 
images; 

means for determining second differences in pixel lumi- 
nance intensity for the second fields of said first and 
second images; 

means for determining a first variance in said first 
differences, and a second variance in said second 
differences; 

means for normalizing said first variance according to a 
mean squared vahic of said first differences to provide 
a fade index for said first fields; 
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means for normalizing said second variance according to 
a mean squared value of said second differences to 
provide a fade index for said second fields; and 

means for comparing an average of the fade indexes to a 
fade threshold, and comparing an average of the mean 
squared values to a motion threshold, to determine if a 
static fade is indicated. 

21. A method for detecting a fade in a current sub-region 
of a current video image in a sequence of digital video 
images, each image including first and second fields, com- 
prising the steps of: 

(a) determining a frame-to-frame residue between the first 
fields of the current sub-region and a correspondingly 
situated sub-region of a reference video image; 

(b) determining a frame-to-frame residue between the 
second fields of the current sub-region and the corre- 
spondingly situated sub-region; 

(c) determining a framc-to-frame variance of the first field 
of the current sub-region according to the associated 
frame-to-frame residue; 

(d) determining a frame-to-frame variance of the second 
field of the current sub-region according to the associ- 
ated frame-to-frame residue; 

(e) averaging the frame-to-frame variances to obtain an 
averaged frame-to-frame variance; 

(f) determining a motion compensation residue betweeo 
the first fields of the current sub-region and a best 
match sub-region of the reference video image; 

(g) determining a motion compensation variance of the 
first field of the current sub-region according to the 30 
associated motion compensation residue; 

(h) determining a motion compensation residue between 
the second fields of the current sub-region and the best 
match sub-region; 

(i) determining a motion compensation variance of the 
second field of the current sub-region according to the 
associated motion compensation residue; 

(j) averaging said motion compensation variances to 
obtain an averaged motion compensation variance; and 

(k) determining if a static fade is indicated for said current 
sub-region according to relative magnitudes of said 
averaged motion compensation variance and said aver- 
aged frame-to-frame variance. 

22. An apparatus for detecting a fade in a current sub- 
region of a current video image in a sequence of digital 
video images, each image including first and second fields, 
comprising: 

(a) means for determining a frame-to-frame residue 
between the first fields of the current sub-region and a 
correspondingly situated sub-region of a reference 
video image; 
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(b) means for determining a frame-to-frame residue 
between the second fields of the current sub-region and 
the correspondingly situated sub-region; 

(c) means for determining a frame-to-frame variance of 
the first field of the current sub-region according to the 
associated frame-to-frame residue; 

(d) means for determining a frame-to-frame variance of 
the second field of the current sub-region according to 
the associated frame-to-frame residue; 

(e) means for averaging the frame-to-frame variances to 
obtain an averaged frame-to-frame variance; 

(£) means for determining a motion compensation residue 
between the first fields of the current sub-region and a 
best match sub-region of the reference video image; 

(g) means for determining a motion compensation vari- 
ance of the first field of the current sub-region accord- 
ing to the associated motion compensation residue; 

(h) means for determining a motion compensation residue 
between the second fields of the current sub-region and 
the best match sub-region; 

(i) means for determining a motion compensation vari- 
ance of the second field of the current sub-region 
according to the associated motion compensation resi- 
due; 

(j) means for averaging said motion compensation vari- 
ances to obtain an averaged motion compensation 
variance; and 

(k) means for determining if a static fade is indicated for 
said current sub-region according to relative magni- 
tudes of said averaged motion compensation variance 
and said averaged frame-to-frame variance. 

23. The method of claim 1, wherein: 

said motion threshold is determined according to a mag- 
nitude of a range of said pixel luminance intensity. 

24. The apparatus of claim 5, wherein: 

said motion threshold is determined according to a mag- 
nitude of a range of said pixel luminance intensity. 

25. The method of claim 19, wherein: 

said motion threshold is determined according to a mag- 
nitude of a range of the pixel luminance intensity of the 
first and second fields. 

26. The apparatus of claim 20, wherein: 

said motion threshold is determined according to a mag- 
nitude of a range of the pixel luminance intensity of the 
first and second fields. 
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